Exercise for Recurrent Neural Networks: PdM Classification


By Prof. Seungchul Lee
http://iai.postech.ac.kr/
Industrial AI Lab at POSTECH

1. Problem Definition

  • Predictive Maintenance
    • This predictive maintenance project focuses on the techniques used to predict when an in-service machine will fail, so that maintenance can be planned in advance.


  • Overview of the RUL estimation strategy
    • The remaining life of a test unit is estimated based on the actual life of a training unit that has the most similar degradation pattern.


  • Data description
    • Training data: It is the aircraft engine run-to-failure data.
    • Testing data: It is the aircraft engine operating data without failure events recorded.
    • Ground truth data: It contains the information of true remaining cycles for each engine in the testing data.



  • Operational conditions
    • Altitude (0 ~ 42,000 feet), Mach number (0 ~ 0.84) and throttle resolver angle (20 ~ 100)
  • 21 sensors
    • Different measurements related to the engine state at runtime.

2. Import Library

In [1]:
import pandas as pd
import numpy as np
import tensorflow as tf
import matplotlib.pyplot as plt
from sklearn import preprocessing
from sklearn.metrics import confusion_matrix
import seaborn as sn

3. Load Data

In [2]:
train_df = pd.read_csv('./data_files/PM_train.txt', sep=" ", header=None)
train_df.head()
Out[2]:
0 1 2 3 4 5 6 7 8 9 ... 18 19 20 21 22 23 24 25 26 27
0 1 1 -0.0007 -0.0004 100.0 518.67 641.82 1589.70 1400.60 14.62 ... 8138.62 8.4195 0.03 392 2388 100.0 39.06 23.4190 NaN NaN
1 1 2 0.0019 -0.0003 100.0 518.67 642.15 1591.82 1403.14 14.62 ... 8131.49 8.4318 0.03 392 2388 100.0 39.00 23.4236 NaN NaN
2 1 3 -0.0043 0.0003 100.0 518.67 642.35 1587.99 1404.20 14.62 ... 8133.23 8.4178 0.03 390 2388 100.0 38.95 23.3442 NaN NaN
3 1 4 0.0007 0.0000 100.0 518.67 642.35 1582.79 1401.87 14.62 ... 8133.83 8.3682 0.03 392 2388 100.0 38.88 23.3739 NaN NaN
4 1 5 -0.0019 -0.0002 100.0 518.67 642.37 1582.85 1406.22 14.62 ... 8133.80 8.4294 0.03 393 2388 100.0 38.90 23.4044 NaN NaN

5 rows × 28 columns

In [3]:
test_df = pd.read_csv('./data_files/PM_test.txt', sep=" ", header=None)
test_df.head()
Out[3]:
0 1 2 3 4 5 6 7 8 9 ... 18 19 20 21 22 23 24 25 26 27
0 1 1 0.0023 0.0003 100.0 518.67 643.02 1585.29 1398.21 14.62 ... 8125.55 8.4052 0.03 392 2388 100.0 38.86 23.3735 NaN NaN
1 1 2 -0.0027 -0.0003 100.0 518.67 641.71 1588.45 1395.42 14.62 ... 8139.62 8.3803 0.03 393 2388 100.0 39.02 23.3916 NaN NaN
2 1 3 0.0003 0.0001 100.0 518.67 642.46 1586.94 1401.34 14.62 ... 8130.10 8.4441 0.03 393 2388 100.0 39.08 23.4166 NaN NaN
3 1 4 0.0042 0.0000 100.0 518.67 642.44 1584.12 1406.42 14.62 ... 8132.90 8.3917 0.03 391 2388 100.0 39.00 23.3737 NaN NaN
4 1 5 0.0014 0.0000 100.0 518.67 642.51 1587.19 1401.92 14.62 ... 8129.54 8.4031 0.03 390 2388 100.0 38.99 23.4130 NaN NaN

5 rows × 28 columns

In [4]:
truth_df = pd.read_csv('./data_files/PM_truth.txt', sep=" ", header=None)
truth_df.head()
Out[4]:
0 1
0 112 NaN
1 98 NaN
2 69 NaN
3 82 NaN
4 91 NaN

3.1. Plot

In [5]:
def plot_sensor_data(data, purpose):
    for j in data.keys().difference([0,1,26,27]).values:

        max_cycle = data.groupby([0]).count()[j].max()
        
        if purpose == 'train':
            plt.figure(figsize = (20, 1.5))
            for i in range(data[0].max()):
                padding = max_cycle - data[data[0] == i][j].values.shape[0]
                plt.plot(np.arange(padding, max_cycle), data[data[0] == i][j].values)
            if j == 2 or j == 3 or j == 4:
                plt.title(str(j-1) + 'th setting')
            else:
                plt.title(str(j-4) + 'th sensor')
            plt.vlines(max_cycle, data[j].min(), data[j].max(), colors = 'r', linestyles = 'dashed')
            plt.text(max_cycle+1, data[j].median(), "failure", fontsize=14)
            plt.show()
            
        if purpose == 'test':
            plt.figure(figsize = (20, 1.5))
            for i in range(data[0].max()):
                plt.plot(data[data[0] == i][j].values)
            if j == 2 or j == 3 or j == 4:
                plt.title(str(j-1) + 'th setting')
            else:
                plt.title(str(j-4) + 'th sensor')
            plt.text(max_cycle+1, data[j].median(), "on working", fontsize=14)
            plt.show()
In [6]:
# plot train data
plot_sensor_data(train_df, 'train')
In [7]:
# plot test data
plot_sensor_data(test_df, 'test')

4. Pre-processing

4.1. Drop NaN column

In [8]:
train_df.drop(train_df.columns[[26, 27]], axis=1, inplace=True)
train_df.columns = ['id', 'cycle', 'setting1', 'setting2', 'setting3', 's1', 's2', 's3',
                     's4', 's5', 's6', 's7', 's8', 's9', 's10', 's11', 's12', 's13', 's14',
                     's15', 's16', 's17', 's18', 's19', 's20', 's21']

test_df.drop(test_df.columns[[26, 27]], axis=1, inplace=True)
test_df.columns = ['id', 'cycle', 'setting1', 'setting2', 'setting3', 's1', 's2', 's3',
                     's4', 's5', 's6', 's7', 's8', 's9', 's10', 's11', 's12', 's13', 's14',
                     's15', 's16', 's17', 's18', 's19', 's20', 's21']

truth_df.drop(truth_df.columns[[1]], axis=1, inplace=True)

4.2. Train Data

In [9]:
train_df.head()
Out[9]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s12 s13 s14 s15 s16 s17 s18 s19 s20 s21
0 1 1 -0.0007 -0.0004 100.0 518.67 641.82 1589.70 1400.60 14.62 ... 521.66 2388.02 8138.62 8.4195 0.03 392 2388 100.0 39.06 23.4190
1 1 2 0.0019 -0.0003 100.0 518.67 642.15 1591.82 1403.14 14.62 ... 522.28 2388.07 8131.49 8.4318 0.03 392 2388 100.0 39.00 23.4236
2 1 3 -0.0043 0.0003 100.0 518.67 642.35 1587.99 1404.20 14.62 ... 522.42 2388.03 8133.23 8.4178 0.03 390 2388 100.0 38.95 23.3442
3 1 4 0.0007 0.0000 100.0 518.67 642.35 1582.79 1401.87 14.62 ... 522.86 2388.08 8133.83 8.3682 0.03 392 2388 100.0 38.88 23.3739
4 1 5 -0.0019 -0.0002 100.0 518.67 642.37 1582.85 1406.22 14.62 ... 522.19 2388.04 8133.80 8.4294 0.03 393 2388 100.0 38.90 23.4044

5 rows × 26 columns

4.2.1. Labeling

In [10]:
train_df['RUL'] = train_df.groupby(['id'])['cycle'].transform(max) - train_df['cycle']
train_df.head()
Out[10]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s13 s14 s15 s16 s17 s18 s19 s20 s21 RUL
0 1 1 -0.0007 -0.0004 100.0 518.67 641.82 1589.70 1400.60 14.62 ... 2388.02 8138.62 8.4195 0.03 392 2388 100.0 39.06 23.4190 191
1 1 2 0.0019 -0.0003 100.0 518.67 642.15 1591.82 1403.14 14.62 ... 2388.07 8131.49 8.4318 0.03 392 2388 100.0 39.00 23.4236 190
2 1 3 -0.0043 0.0003 100.0 518.67 642.35 1587.99 1404.20 14.62 ... 2388.03 8133.23 8.4178 0.03 390 2388 100.0 38.95 23.3442 189
3 1 4 0.0007 0.0000 100.0 518.67 642.35 1582.79 1401.87 14.62 ... 2388.08 8133.83 8.3682 0.03 392 2388 100.0 38.88 23.3739 188
4 1 5 -0.0019 -0.0002 100.0 518.67 642.37 1582.85 1406.22 14.62 ... 2388.04 8133.80 8.4294 0.03 393 2388 100.0 38.90 23.4044 187

5 rows × 27 columns

4.2.2. Normalize

In [11]:
cols_normalize = train_df.columns.difference(['id','cycle','RUL'])
cols_normalize
Out[11]:
Index(['s1', 's10', 's11', 's12', 's13', 's14', 's15', 's16', 's17', 's18',
       's19', 's2', 's20', 's21', 's3', 's4', 's5', 's6', 's7', 's8', 's9',
       'setting1', 'setting2', 'setting3'],
      dtype='object')
In [12]:
# normalize
min_max_scaler = preprocessing.MinMaxScaler()
norm_train_df = pd.DataFrame(min_max_scaler.fit_transform(train_df[cols_normalize]),
                             columns=cols_normalize,
                             index=train_df.index)

norm_train_df.head()
Out[12]:
s1 s10 s11 s12 s13 s14 s15 s16 s17 s18 ... s3 s4 s5 s6 s7 s8 s9 setting1 setting2 setting3
0 0.0 0.0 0.369048 0.633262 0.205882 0.199608 0.363986 0.0 0.333333 0.0 ... 0.406802 0.309757 0.0 1.0 0.726248 0.242424 0.109755 0.459770 0.166667 0.0
1 0.0 0.0 0.380952 0.765458 0.279412 0.162813 0.411312 0.0 0.333333 0.0 ... 0.453019 0.352633 0.0 1.0 0.628019 0.212121 0.100242 0.609195 0.250000 0.0
2 0.0 0.0 0.250000 0.795309 0.220588 0.171793 0.357445 0.0 0.166667 0.0 ... 0.369523 0.370527 0.0 1.0 0.710145 0.272727 0.140043 0.252874 0.750000 0.0
3 0.0 0.0 0.166667 0.889126 0.294118 0.174889 0.166603 0.0 0.333333 0.0 ... 0.256159 0.331195 0.0 1.0 0.740741 0.318182 0.124518 0.540230 0.500000 0.0
4 0.0 0.0 0.255952 0.746269 0.235294 0.174734 0.402078 0.0 0.416667 0.0 ... 0.257467 0.404625 0.0 1.0 0.668277 0.242424 0.149960 0.390805 0.333333 0.0

5 rows × 24 columns

In [13]:
join_df = train_df[train_df.columns.difference(cols_normalize)].join(norm_train_df)

train_df = join_df.reindex(columns = train_df.columns)

train_df.head()
Out[13]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s13 s14 s15 s16 s17 s18 s19 s20 s21 RUL
0 1 1 0.459770 0.166667 0.0 0.0 0.183735 0.406802 0.309757 0.0 ... 0.205882 0.199608 0.363986 0.0 0.333333 0.0 0.0 0.713178 0.724662 191
1 1 2 0.609195 0.250000 0.0 0.0 0.283133 0.453019 0.352633 0.0 ... 0.279412 0.162813 0.411312 0.0 0.333333 0.0 0.0 0.666667 0.731014 190
2 1 3 0.252874 0.750000 0.0 0.0 0.343373 0.369523 0.370527 0.0 ... 0.220588 0.171793 0.357445 0.0 0.166667 0.0 0.0 0.627907 0.621375 189
3 1 4 0.540230 0.500000 0.0 0.0 0.343373 0.256159 0.331195 0.0 ... 0.294118 0.174889 0.166603 0.0 0.333333 0.0 0.0 0.573643 0.662386 188
4 1 5 0.390805 0.333333 0.0 0.0 0.349398 0.257467 0.404625 0.0 ... 0.235294 0.174734 0.402078 0.0 0.416667 0.0 0.0 0.589147 0.704502 187

5 rows × 27 columns

4.3. Test Data

4.3.1. Labeling

In [14]:
# Get value of max cycle
rul = pd.DataFrame(test_df.groupby('id')['cycle'].max()).reset_index()
rul.columns = ['id', 'max']
rul.head()
Out[14]:
id max
0 1 31
1 2 49
2 3 126
3 4 106
4 5 98
In [15]:
truth_df.columns = ['more']
truth_df['id'] = truth_df.index + 1
truth_df.head()
Out[15]:
more id
0 112 1
1 98 2
2 69 3
3 82 4
4 91 5
In [16]:
truth_df['rtf'] = truth_df['more'] + rul['max']
truth_df.drop('more', axis = 1, inplace = True)
truth_df.head()
Out[16]:
id rtf
0 1 143
1 2 147
2 3 195
3 4 188
4 5 189
In [17]:
# Get RUL
test_df = test_df.merge(truth_df)

test_df['RUL'] = test_df['rtf'] - test_df['cycle']
test_df.drop('rtf', axis = 1, inplace = True)

test_df.head()
Out[17]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s13 s14 s15 s16 s17 s18 s19 s20 s21 RUL
0 1 1 0.0023 0.0003 100.0 518.67 643.02 1585.29 1398.21 14.62 ... 2388.03 8125.55 8.4052 0.03 392 2388 100.0 38.86 23.3735 142
1 1 2 -0.0027 -0.0003 100.0 518.67 641.71 1588.45 1395.42 14.62 ... 2388.06 8139.62 8.3803 0.03 393 2388 100.0 39.02 23.3916 141
2 1 3 0.0003 0.0001 100.0 518.67 642.46 1586.94 1401.34 14.62 ... 2388.03 8130.10 8.4441 0.03 393 2388 100.0 39.08 23.4166 140
3 1 4 0.0042 0.0000 100.0 518.67 642.44 1584.12 1406.42 14.62 ... 2388.05 8132.90 8.3917 0.03 391 2388 100.0 39.00 23.3737 139
4 1 5 0.0014 0.0000 100.0 518.67 642.51 1587.19 1401.92 14.62 ... 2388.03 8129.54 8.4031 0.03 390 2388 100.0 38.99 23.4130 138

5 rows × 27 columns

In [18]:
# Labeling
period = 30

train_df['label_bc'] = train_df['RUL'].apply(lambda x: 1 if x <= period else 0)
test_df['label_bc'] = test_df['RUL'].apply(lambda x: 1 if x <= period else 0)
train_df.head()
Out[18]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s14 s15 s16 s17 s18 s19 s20 s21 RUL label_bc
0 1 1 0.459770 0.166667 0.0 0.0 0.183735 0.406802 0.309757 0.0 ... 0.199608 0.363986 0.0 0.333333 0.0 0.0 0.713178 0.724662 191 0
1 1 2 0.609195 0.250000 0.0 0.0 0.283133 0.453019 0.352633 0.0 ... 0.162813 0.411312 0.0 0.333333 0.0 0.0 0.666667 0.731014 190 0
2 1 3 0.252874 0.750000 0.0 0.0 0.343373 0.369523 0.370527 0.0 ... 0.171793 0.357445 0.0 0.166667 0.0 0.0 0.627907 0.621375 189 0
3 1 4 0.540230 0.500000 0.0 0.0 0.343373 0.256159 0.331195 0.0 ... 0.174889 0.166603 0.0 0.333333 0.0 0.0 0.573643 0.662386 188 0
4 1 5 0.390805 0.333333 0.0 0.0 0.349398 0.257467 0.404625 0.0 ... 0.174734 0.402078 0.0 0.416667 0.0 0.0 0.589147 0.704502 187 0

5 rows × 28 columns

4.3.2. Normalize

In [19]:
norm_test_df = pd.DataFrame(min_max_scaler.transform(test_df[cols_normalize]), 
                            columns=cols_normalize, 
                            index=test_df.index)

test_join_df = test_df[test_df.columns.difference(cols_normalize)].join(norm_test_df)
test_df = test_join_df.reindex(columns = test_df.columns)
test_df = test_df.reset_index(drop=True)

test_df.head()
Out[19]:
id cycle setting1 setting2 setting3 s1 s2 s3 s4 s5 ... s14 s15 s16 s17 s18 s19 s20 s21 RUL label_bc
0 1 1 0.632184 0.750000 0.0 0.0 0.545181 0.310661 0.269413 0.0 ... 0.132160 0.308965 0.0 0.333333 0.0 0.0 0.558140 0.661834 142 0
1 1 2 0.344828 0.250000 0.0 0.0 0.150602 0.379551 0.222316 0.0 ... 0.204768 0.213159 0.0 0.416667 0.0 0.0 0.682171 0.686827 141 0
2 1 3 0.517241 0.583333 0.0 0.0 0.376506 0.346632 0.322248 0.0 ... 0.155640 0.458638 0.0 0.416667 0.0 0.0 0.728682 0.721348 140 0
3 1 4 0.741379 0.500000 0.0 0.0 0.370482 0.285154 0.408001 0.0 ... 0.170090 0.257022 0.0 0.250000 0.0 0.0 0.666667 0.662110 139 0
4 1 5 0.580460 0.500000 0.0 0.0 0.391566 0.352082 0.332039 0.0 ... 0.152751 0.300885 0.0 0.166667 0.0 0.0 0.658915 0.716377 138 0

5 rows × 28 columns

4.4. Data Generation

In [20]:
# function to generate input sequences
def gen_sequence(id_df, n_step, seq_cols):
    df_zeros = pd.DataFrame(np.zeros((n_step - 1, id_df.shape[1])), columns = id_df.columns) # zero padding
    id_df = df_zeros.append(id_df, ignore_index = True) # zero padding
    data_array = id_df[seq_cols].values
    num_elements = data_array.shape[0]
    lstm_array = []
    for start, stop in zip(range(0, num_elements - n_step), range(n_step, num_elements)):
        lstm_array.append(data_array[start:stop, :])
    return np.array(lstm_array)

# function to generate labels
def gen_label(id_df, n_step, seq_cols, label):
    df_zeros=pd.DataFrame(np.zeros((n_step - 1, id_df.shape[1])), columns = id_df.columns) # zero padding
    id_df=df_zeros.append(id_df, ignore_index = True) # zero padding
    data_array = id_df[seq_cols].values
    num_elements = data_array.shape[0]
    y_label=[]
    for start, stop in zip(range(0, num_elements - n_step), range(n_step, num_elements)):
        y_label.append(id_df[label][stop])
    return np.array(y_label)
In [21]:
# generate data
n_step = 50
n_sensor = 24

sensor_cols = ['s' + str(i) for i in range(1,22)]
x_cols = ['setting1', 'setting2', 'setting3']
x_cols.extend(sensor_cols)

X_train = np.concatenate(list(list(gen_sequence(train_df[train_df['id'] == id], n_step, x_cols)) for id in train_df['id'].unique()))
y_train = np.concatenate(list(list(gen_label(train_df[train_df['id'] == id], n_step, x_cols,'label_bc')) for id in train_df['id'].unique()))
X_test=np.concatenate(list(list(gen_sequence(test_df[test_df['id']==id], n_step, x_cols)) for id in test_df['id'].unique()))
y_test=np.concatenate(list(list(gen_label(test_df[test_df['id']==id], 50, x_cols,'label_bc')) for id in test_df['id'].unique()))

y_train = y_train[:,np.newaxis]
y_test = y_test[:,np.newaxis]

print(X_train.shape)
print(y_train.shape)
print(X_test.shape)
print(y_test.shape)
(20531, 50, 24)
(20531, 1)
(12996, 50, 24)
(12996, 1)

5. Build a Model

5.1. 1D Convolution



  • Eventhough input is 2D ex) 20x14, output-shape is not 2D , but 1D Matrix
  • What if we want to train N filters (N is number of filters)? then output shape is (stacked 1D) 2D = 1D x N matrix


In [22]:
# define the CNN structure and placeholder
input_length = n_step
input_ch = n_sensor

n_hidden = ?
n_output = ?

x = tf.placeholder(tf.float32, [None, input_length, input_ch])
y = tf.placeholder(tf.float32, [None, n_output])
In [23]:
# build a model
def net(x):
    ## First convolution layer
    conv1 = tf.layers.conv1d(inputs = x, 
                              filters = ?,
                              kernel_size = ?,
                              padding = 'SAME',
                              activation = tf.nn.relu)
    maxp1 = tf.layers.max_pooling1d(inputs = conv1,
                                    pool_size = 2,
                                    strides = 2)

    ## Second convolution layer
    conv2 = tf.layers.conv1d(inputs = maxp1,
                              filters = ?,
                              kernel_size = ?,
                              padding = 'SAME',
                              activation = tf.nn.relu)
    maxp2 = tf.layers.max_pooling1d(inputs = conv2,
                                    pool_size = 2,
                                    strides = 2)

    maxp2_re = tf.contrib.layers.flatten(maxp2)
    
    ### Fully connected (= dense connected)
    hidden = tf.layers.dense(inputs = maxp2_re,
                             units = n_hidden,
                             activation = tf.nn.relu)
    output = tf.layers.dense(inputs = hidden,
                             units = n_output)

    return output
In [24]:
# define loss and optimizer
LR = 0.0005

pred = net(x)
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(labels = y, logits = pred))

optm = tf.train.AdamOptimizer(LR).minimize(loss)
WARNING: Logging before flag parsing goes to stderr.
W0128 23:04:44.895468 21892 deprecation.py:323] From <ipython-input-23-21154d820bb0>:8: conv1d (from tensorflow.python.layers.convolutional) is deprecated and will be removed in a future version.
Instructions for updating:
Use `tf.keras.layers.Conv1D` instead.
W0128 23:04:44.899456 21892 deprecation.py:506] From c:\users\juwonna7\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\init_ops.py:1251: calling VarianceScaling.__init__ (from tensorflow.python.ops.init_ops) with dtype is deprecated and will be removed in a future version.
Instructions for updating:
Call initializer instance with the dtype argument instead of passing it to the constructor
W0128 23:04:44.981238 21892 deprecation.py:323] From <ipython-input-23-21154d820bb0>:11: max_pooling1d (from tensorflow.python.layers.pooling) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.MaxPooling1D instead.
W0128 23:04:46.075379 21892 lazy_loader.py:50] 
The TensorFlow contrib module will not be included in TensorFlow 2.0.
For more information, please see:
  * https://github.com/tensorflow/community/blob/master/rfcs/20180907-contrib-sunset.md
  * https://github.com/tensorflow/addons
  * https://github.com/tensorflow/io (for I/O related ops)
If you depend on functionality not listed there, please file an issue.

W0128 23:04:46.076404 21892 deprecation.py:323] From c:\users\juwonna7\appdata\local\programs\python\python36\lib\site-packages\tensorflow\contrib\layers\python\layers\layers.py:1634: flatten (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.flatten instead.
W0128 23:04:46.237944 21892 deprecation.py:323] From <ipython-input-23-21154d820bb0>:28: dense (from tensorflow.python.layers.core) is deprecated and will be removed in a future version.
Instructions for updating:
Use keras.layers.dense instead.
W0128 23:04:46.455363 21892 deprecation.py:323] From c:\users\juwonna7\appdata\local\programs\python\python36\lib\site-packages\tensorflow\python\ops\nn_impl.py:180: add_dispatch_support.<locals>.wrapper (from tensorflow.python.ops.array_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.where in 2.0, which has the same broadcast rule as np.where

6. Batch Maker

In [25]:
n_batch = 64

dataset_train = tf.data.Dataset.from_tensor_slices((X_train, y_train))
dataset_train = dataset_train.shuffle(100000).repeat().batch(n_batch)
iterator_train = dataset_train.make_one_shot_iterator()
next_batch_train = iterator_train.get_next()

dataset_test = tf.data.Dataset.from_tensor_slices((X_test, y_test))
dataset_test = dataset_test.shuffle(100000).repeat().batch(n_batch)
iterator_test = dataset_test.make_one_shot_iterator()
next_batch_test = iterator_test.get_next()
W0128 23:04:47.940392 21892 deprecation.py:323] From <ipython-input-25-39ad9ad95a60>:5: DatasetV1.make_one_shot_iterator (from tensorflow.python.data.ops.dataset_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `for ... in dataset:` to iterate over a dataset. If using `tf.estimator`, return the `Dataset` object directly from your input function. As a last resort, you can use `tf.compat.v1.data.make_one_shot_iterator(dataset)`.

7. Optimize

In [26]:
sess = tf.Session()
sess.run(tf.global_variables_initializer())

n_iter = 1500
n_prt = 100

for i in range(n_iter):
    train_x, train_y = sess.run(next_batch_train)
    
    sess.run(optm, feed_dict = {x: train_x, y: train_y})
    c = sess.run(loss, feed_dict = {x: train_x, y: train_y})
    
    if i % n_prt == 0:
        print("Iter: {}".format(i), "Cost: {}".format(c))
Iter: 0 Cost: 0.6619477272033691
Iter: 100 Cost: 0.3490923047065735
Iter: 200 Cost: 0.20090141892433167
Iter: 300 Cost: 0.0922442376613617
Iter: 400 Cost: 0.11518950760364532
Iter: 500 Cost: 0.06417232751846313
Iter: 600 Cost: 0.033545296639204025
Iter: 700 Cost: 0.05708543583750725
Iter: 800 Cost: 0.06106068938970566
Iter: 900 Cost: 0.026056740432977676
Iter: 1000 Cost: 0.05036859214305878
Iter: 1100 Cost: 0.11891248822212219
Iter: 1200 Cost: 0.22156760096549988
Iter: 1300 Cost: 0.03762578219175339
Iter: 1400 Cost: 0.043580561876297

8. Test or Evaluation

In [27]:
test_pred = sess.run(pred, feed_dict = {x: X_test})
test_pred = 1/(1 + np.exp(-test_pred))

accuracy = np.mean(np.equal(y_test > 0.5, test_pred > 0.5))
print('Accuracy: {0:.2f} %'.format(accuracy * 100))
Accuracy: 99.12 %
In [28]:
# confusion matrix
pd.crosstab((y_test[:,0] > 0.5) * 1, (test_pred[:,0] > 0.5) * 1, rownames = ['True'], colnames = ['Predicted'], margins = True)
Out[28]:
Predicted 0 1 All
True
0 12615 49 12664
1 65 267 332
All 12680 316 12996
In [29]:
def prob_failure(machine_id):
    machine_df = test_df[test_df['id'] == machine_id]
    machine_test = gen_sequence(machine_df, n_step, x_cols)

    test_pred = sess.run(pred, feed_dict = {x: machine_test})
    test_pred = 1/(1 + np.exp(-test_pred))
    
    failure_prob = list(test_pred[-1] * 100)[0]
    return failure_prob
In [30]:
machine_id = 49
print('Probability that machine will fail within 30 days: {0:.2f} %'.format(prob_failure(machine_id)))
Probability that machine will fail within 30 days: 99.44 %